Vwdre – a Vision-based Approach for Mining Data from Search Engine Result Pages

The data extraction from the dynamically generated web pages is a challenging factor because the result of the search engines are always different for every query submitted. Many techniques were proposed to address this issue but most of them have the common problem of language-dependency. In order to overcome the limitations of previous works, there are few ways which analyze visual features of the web page. In this paper, we proposed a new vision-based approach which is independent of the code used. It broadly utilizes the visual features on the search engine result pages to locate the data region so asto mine the data records from it. We develop a clustering by similarity algorithm to check the similarity of data records. Also, we propose a technique to generate the wrapper for data record extraction by examining the multiple result pages from the same search engine.